The Integration of Textual and Visual Search in Image Databases∗

Image (and video) databases have been studied mainly from the point of view of Content Based retrieval. One problem of CBIR has always been the “semantic gap” between the level of abstraction at which a user asks certain query and the level of abstraction at which information can be extracted from images. Labeling images as a means of attaching more abstract semantics to them is an obvious solution, but it is recognized to have two drawbacks: (1) it is a manual and very expensive procedure, and (2) it can’t capture all the possible semantics that a user can attach to an image. This paper proposes a technique called “partial labeling” by which a fraction of the database is labeled using non-specific labels, and the results of the search in the labeled part is used as a “seed” to do a visual search on the whole database.